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Why GAO Did This Study 

Mandated to screen all checked 
baggage by using explosive 
detection systems at airports by 
December 31, 2003, the 
Transportation Security 
Administration (TSA) has deployed 
two types of screening equipment: 
explosive detection systems (EDS), 
which use computer-aided 
tomography X-rays to recognize 
explosives, and explosive trace 
detection (ETD) systems, which 
use chemical analysis to detect 
explosive residues. This report 
discusses (1) EDS and ETD 
maintenance costs, (2) factors that 
played a role in these costs, and (3) 
the extent to which TSA conducts 
oversight of maintenance 
contracts. GAO reviewed TSA's 
contract files and processes for 
reviewing contractor cost and 
performance data. 



What GAO Recommends 



GAO recommends that the 
Secretary of Homeland Security 
direct TSA to (1) establish a 
timeline to close out the contract 
with Boeing Service Company 
(Boeing) and report to 
congressional committees on 
actions to recover any excessive 
fees awarded to Boeing, (2) 
establish a timeline to complete the 
EDS life-cycle model, and (3) revise 
policies to require documentation 
for monitoring EDS and ETD 
maintenance contracts. The 
Department of Homeland Security 
concurred with GAO's 
recommendations and described 
actions TSA had taken or planned 
to take to implement them. 
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What GAO Found 

TSA obligated almost $470 million from fiscal years 2002 through 2005 for 
EDS and ETD maintenance, according to TSA budget documents. In fiscal 
year 2006, TSA estimates it will spend $199 million and has projected it will 
spend $234 million in fiscal year 2007. TSA was not able to provide GAO with 
data on the maintenance cost per machine before fiscal year 2005 because, 
according to TSA officials, its previous contract with Boeing to install and 
maintain EDS and ETD machines was not structured to capture these data. 

Several factors have played a role in EDS and ETD maintenance costs. 
According to a September 2004 Department of Homeland Security's Office of 
Inspector General report, TSA did not follow sound contracting practices in 
administering the contract with Boeing, and TSA paid provisional award fees 
totaling $44 million through December 2003 without any evaluation of 
Boeing's performance. TSA agreed to recover any excessive award fees paid 
to Boeing if TSA determined that such fees were not warranted. In 
responding to our draft report, DHS told us that TSA and Boeing had 
reached an agreement in principle on this matter and that documentation 
was in the approval process with closure anticipated in July 2006. Moreover, 
TSA did not develop life-cycle cost models before any of the maintenance 
contracts were executed and, as a result, TSA does not have a sound 
estimate of maintenance costs for all the years the machines are expected to 
be in operation. DHS also stated in its comments on our draft report that a 
TSA contractor expected to complete a prototype life-cycle cost model by 
September 2006 and that TSA anticipated that the EDS model would be 
completed 12 months after the prototype was approved. Without such an 
analysis, TSA may not be identifying cost efficiencies and making informed 
procurement decisions on future purchases of EDS and ETD machines and 
maintenance contracts. TSA has taken actions to control costs, such as 
entering into firm-fixed-price contracts for maintenance starting in March 
2005, which have advantages to the government because price certainty is 
guaranteed. Further, TSA incorporated standard performance requirements 
in the contracts including metrics related to machine reliability and monthly 
performance reviews. For EDS contractors, TSA has specified that the full 
agreed price would be paid only if mean downtime (i.e., the number of hours 
a machine is out of service in a month divided by the number of times that 
machine is out of service per month) requirements are met. 

Although TSA has policies for monitoring contracts, TSA officials provided 
no evidence that they are reviewing required contractor-submitted 
performance data, such as mean downtime data. TSA officials told GAO that 
they perform such reviews, but do not document their activities because 
there are no TSA policies and procedures requiring them to do so. As a 
result, without adequate documentation, TSA does not have reasonable 
assurance that contractors are performing as required and that full payment 
is justified based on meeting mean downtime requirements. 



United States Government Accountability Office 



Contents 



Letter 




1 




Results in Brief 
Fto pk"0roi in H 

Ud^ JVfiiA KJ 111 LU 

Results 

Conclusions 

Recommendations 

Agency Comments and Our Evaluation 


3 
a 

8 
14 
15 
15 


Appendix I 


Information for Congressional Committees 


17 


Appendix II 


Agency Comments 


47 


Appendix III 


GAO Contact and Staff Acknowledgements 


51 


Tables 








Table 1: Number of EDS and ETD Machines and Annual Per- 

Machine Maintenance Cost, Fiscal Year 2005 and Fiscal 
Year 2006 

Table 2: Mean Downtime Requirement for EDS Contractors, 2005 
through 2009 


9 
12 


Figure 








Figure 1: EDS and ETD Machines Used by TSA to Screen Checked 
Baggage 


7 



This is a work of the U.S. government and is not subject to copyright protection in the 
United States. It may be reproduced and distributed in its entirety without further 
permission from GAO. However, because this work may contain copyrighted images or 
other material, permission from the copyright holder may be necessary if you wish to 
reproduce this material separately. 



Page i 



GAO-06-795 TSA EDS Maintenance Costs 



i 

^ G A O 

— Accountability * Integrity * Reliability 

United States Government Accountability Office 
Washington, DC 20548 



July 31, 2006 

The Honorable Judd Gregg 
Chairman 

The Honorable Robert C. Byrd 
Ranking Member 

Subcommittee on Homeland Security 
Committee on Appropriations 
United States Senate 

The Honorable Harold Rogers 
Chairman 

The Honorable Martin Olav Sabo 
Ranking Member 

Subcommittee on Homeland Security 
Committee on Appropriations 
House of Representatives 

After the terrorist attacks of September 11, 2001, which highlighted the 
vulnerability of U.S. aircraft to acts of terrorism, Congress passed and the 
President signed into law the Aviation and Transportation Security Act 
(ATSA), mandating, among other things, that all checked baggage at U.S. 
airports be screened using explosive detection systems by December 31, 
2002. 1 To meet this requirement, the Transportation Security 
Administration (TSA) deployed two types of equipment to screen checked 
baggage for explosives: (1) explosive detection systems (EDS) that use 
specialized X-rays to detect characteristics of explosives that may be 
contained in baggage as it moves along a conveyor belt and (2) explosive 
trace detection (ETD) machines, whereby an individual (i.e., a baggage 
screener, or transportation security officer) swabs baggage and then 



Aviation and Transportation Security Act, Pub. L. No. 107-71, § 110(b), 115 Stat. 597, 615 
(2001). Section 425 of the subsequently enacted Homeland Security Act of 2002, Pub. L. No. 
107-296, 116 Stat. 2135, 2185-86, in effect, extended this mandate to December 31, 2003. See 
49 U.S.C. § 44901(d). 
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inserts the swab into the ETD machine, which in turn can detect chemical 
residues that may indicate the presence of explosives within a bag. 2 

Pursuant to ATSA, TSA assumed operational responsibility for conducting 
the screening of checked baggage, which includes the procurement, 
installation, and maintenance of EDS and ETD machines. By the end of 
fiscal year 2006, TSA will have deployed over 1,400 EDS and 6,600 ETD 
machines at baggage-screening locations in over 400 airports nationwide, 
according to TSA budget documents. TSA has used contractors to perform 
preventative and corrective maintenance on these EDS and ETD 
machines. 

House Conference Report 109-241, which accompanied the Department of 
Homeland Security Appropriations Act, 2006 (Public Law 109-90) directed 
that we report on the reasons for cost increases in maintaining TSA's 
explosive detection systems, including TSA's contracting practices that 
may have affected cost increases. 3 With regard to this requirement, on 
April 19, 2006, we provided you with information on the results of our 
review (see app. I), which has been updated as appropriate in this report. 
Subsequently, in May 2006, the House Appropriations Committee stated in 
its report accompanying the Department of Homeland Security 
Appropriations Bill for fiscal year 2007 its long-standing concerns about 
the increasing costs for EDS and ETD maintenance and specifically, TSA's 
recovery of any excess award fees from a previous EDS and ETD 
contractor. 4 Further, the report stated that TSA should submit a report to 
the House Appropriations Committee on any actions it has taken to collect 
excessive award fees, how much has been received to date, and specific 
plans to obligate these collections. 

This report addresses the following questions: 

• What are the historical, current, and projected costs for the 
maintenance of EDS and ETD machines? 



2 To satisfy the ATSA mandate, TSA interpreted the term explosive detection system to 
include both explosive detection systems (EDS) and explosive trace detection (ETD) 
machines. 

3 See H.R. Conf. Rep. No. 109-241, at 52 (2005). 

"Department of Homeland Security Appropriations Act, 2007, H.R. 5441, 109th Cong. (2006); 
H.R. Rep. No. 109-476, tit. II, at 49-50 (2006). 
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• What factors played a role in EDS and ETD maintenance costs, and 
what factors could affect future costs? 

• What has TSA done to control EDS and ETD maintenance costs? 

• To what extent does TSA oversee the performance of EDS and ETD 
maintenance contractors? 

To address our objectives, we reviewed TSA contract files and related cost 
data, TSA processes for reviewing contract performance data, budget 
documents for fiscal years 2003 through 2007, acquisition and strategic 
plans, a Department of Homeland Security's Office of the Inspector 
General (DHS OIG) report, 5 GAO standards for internal controls, 6 and our 
previous work on TSA's acquisition function. 7 We interviewed TSA 
headquarters officials, DHS OIG officials, and EDS and ETD contractor 
representatives. For purposes of our review, we focused on amounts 
obligated under contracts to maintain the machines. We did not review 
TSA's negotiations for maintenance services or the process for awarding 
contracts. Nor did we assess other direct or indirect costs that may have 
been related to TSA or DHS employees engaged in contract administration 
or other related items. 

We performed our work from January 2006 through July 2006 in 
accordance with generally accepted government auditing standards. 



Rpsillts ID Rripf According to TSA budget documents, TSA obligated almost $470 million 

xves uiis 11 di lei from fisca] year 20()2 through flgcal year 20Q5 for EDg and ETD 

maintenance. In fiscal year 2006, TSA estimates it will spend $199 million 



5 U.S. Department of Homeland Security, Office of Inspector General, Evaluation of TSA's 
Contract for the Installation and Maintenance of Explosive Detection Equipment at 
United States Airports, OIG-04-44 (Washington, D.C.: September 2004). 

6 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 
(Washington, D.C.: November 1999). 

7 GAO, Transportation Security Administration: High-Level Attention Needed to 
Strengthen Acquisition Function, GAO-04-544 (Washington, D.C.: May 2004). 
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and has projected it will spend $234 million in fiscal year 2007. 8 TSA did 
not provide us with projections of EDS and ETD maintenance costs 
beyond 2007. TSA officials told us that future EDS and ETD maintenance 
costs will be influenced by the number, type, quantity, and locations of 
machines necessary to support system configurations at airports and on 
decisions related to the deployment of new technologies and the 
refurbishment of existing equipment, among other things. The current 
contracts have negotiated maintenance prices per machine through March 
2009, if TSA decides to exercise option years in the contracts. 

Different factors played a role in EDS and ETD maintenance costs. 
According to a September 2004 DHS OIG report, 9 TSA did not follow 
sound contracting practices in administering the contract with Boeing 
Service Company (Boeing) that was primarily for the installation and 
maintenance of EDS and ETD machines from June 2002 through March 
2005. According to DHS OIG officials, TSA's failure to control costs under 
the Boeing contract contributed to increases in maintenance costs. Among 
other things, the DHS OIG report stated that TSA has paid provisional 
award fees totaling $44 million through December 2003 without any 
evaluation of Boeing's performance. In response to the DHS OIG, TSA 
agreed to recover any excessive award fees paid to Boeing, if TSA 
determined that such fees were not warranted. In responding to our draft 
report, DHS told us that TSA and Boeing have reached an agreement in 
principle on this matter and the documentation is in the approval process 
with closure anticipated in July 2006. Moreover, TSA did not develop life- 
cycle cost estimates before any of the maintenance contracts were 
executed, and, as a result, TSA did not have a sound estimate of 
maintenance costs for all the years the machines are expected to be in 
operation. DHS also stated in its comments on our draft report that a TSA 
contractor expects to complete a prototype life-cycle cost model by 
September 2006 and that TSA anticipated that the EDS model would be 
completed 12 months after the prototype was approved. Without such an 
analysis, TSA may not be identifying cost efficiencies and making 



Amounts attributed to maintenance also include utility costs, such as electricity, that 
generally amount to less than 10 percent of the overall amount allocated for maintenance 
each fiscal year, according to TSA officials. Further, TSA officials told us they could 
provide us with amounts obligated for fiscal years 2005 and 2006, but could not provide us 
with the amounts expended for this time period. 

9 OIG-04-44. 
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informed procurement decisions on future purchases of EDS and ETD 
machines and maintenance contracts. 

TSA has taken several actions to control EDS and ETD maintenance costs, 
such as entering into firm-fixed-price contracts starting in March 2005, 
which have certain advantages to the government because price certainty 
is guaranteed. Also, TSA included several performance requirements in the 
current contracts, including metrics related to machine reliability, 
maintainability, and availability, and specific cost data related to 
maintenance and repair, and required monthly performance reviews. For 
EDS contractors, TSA also incorporated provisions to specify that the full 
agreed price will be paid only if mean downtime requirements 10 are met. 

Although TSA has policies for monitoring contracts, TSA officials provided 
no evidence that they are reviewing required contractor-submitted 
performance data, such a mean downtime data. TSA officials told GAO 
that they perform such reviews, but do not document their activities 
because there are not TSA policies and procedures requiring them to do 
so. As a result, without adequate documentation, TSA does not have 
reasonable assurance that contractors are performing as required and that 
full payment is justified based on meeting mean downtime requirements. 

We are recommending that the Secretary of Homeland Security direct TSA 
to establish a timeline to close out the Boeing contract and report to 
congressional committees on its actions to recover any excessive fees 
awarded to Boeing; establish a timeline for completing life-cycle cost 
models for EDS, which TSA recently began; and revise policies and 
procedures to require documentation of the monitoring of EDS and ETD 
maintenance contracts to provide reasonable assurance that contractor 
maintenance cost data and performance data are recorded and reported in 
accordance with TSA contractual requirements and that self-reported 
contractor mean downtime data are valid, reliable, and justify the full 
payment of the contract amount. 

We provided a draft of this report to DHS for review. DHS, in its written 
comments, concurred with our findings and recommendations and 



Mean downtime is a performance requirement in EDS and ETD maintenance contracts. 
Mean downtime is calculated by the number of hours a machine is out of service in a 
month divided by the number of times that machine is out of service per month. For 
example, if a machine has a total downtime of 50 hours per month and is out of service 5 
times in that month, the MDT would be equal to 50 divided by 5, which is 10 hours. 
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described actions that it has initiated or plans to take to address the issues 
identified. For a reprint of DHS's comments, see appendix II. 



Background 



With the passage of the Aviation and Transportation Security Act (ATSA) 
in November 2001, TSA assumed from the Federal Aviation Administration 
(FAA) the majority of the responsibility for civil aviation security, 
including the commercial aviation system. 11 ATSA required that TSA 
screen 100 percent of checked baggage using explosive detection systems 
by December 31, 2002. As it became apparent that certain airports would 
not meet the December 2002 deadline, the Homeland Security Act of 2002 
in effect extended the deadline to December 31, 2003, for noncompliant 
airports. 12 Under ATSA, TSA is responsible for the procurement, 
installation, and maintenance of explosive detection systems used to 
screen checked baggage for explosives. Airport operators and air carriers 
continued to be responsible for processing and transporting passengers' 
checked baggage from the check-in counter to the airplane. 

Explosive detection systems include EDS and ETD machines (fig. 1). EDS 
uses computer-aided tomography X-rays adapted from the medical field to 
automatically recognize the characteristic signatures of threat explosives. 
By taking the equivalent of hundreds of X-ray pictures of a bag from 
different angles, EDS examines the objects inside of the baggage to 
identify characteristic signatures of threat explosives. TSA has certified, 
procured, and deployed EDS manufactured by three companies — L-3 
Communications Security and Detection Systems (L-3); General Electric 
InVision, Inc. 13 (GE InVision); and Reveal Imaging Technologies, Inc. 
(Reveal). In general, EDS is used for checked baggage screening. ETD 
machines work by detecting vapors and residues of explosives. Human 
operators collect samples by rubbing bags with swabs, which are then 
chemically analyzed in the ETD machine to identify any traces of 
explosive materials. ETD machines are used for both checked baggage and 
passenger carry-on baggage screening. TSA has certified, procured, and 
deployed ETD machines from three manufacturers, Thermo Electron 
Corporation, Smiths Detection, and General Electric Company. 



'Pub. L. No. 107-71, § 101, 115 Stat, at 597. See 49 U.S.C. § 114(d). 

! Pub. L. No. 107-296, § 425, 116 Stat, at 2185-86. See 49 U.S.C. § 44901(d). 

1 General Electric InVision, Inc. is an entity of General Electric Company. 
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Figure 1 : EDS and ETD Machines Used by TSA to Screen Checked Baggage 



EDS machine 

Uses computerized 
tomography X-rays to 
recognize the 
characteristic signatures 
of explosives. 




ETD machine 

Uses chemical 
analysis to detect 
traces of explosive 
material vapors or 
residues. 



Source: GAO. 



TSA's EDS and ETD maintenance contracts provide for preventative and 
corrective maintenance. Preventative maintenance includes scheduled 
activities, such as changing filters or cleaning brushes, to increase 
machine reliability and are performed monthly, quarterly, or yearly based 
on the contractors' maintenance schedules. Corrective maintenance 
includes actions performed to restore machines to operating condition 
after failure, such as repairing the conveyer belt mechanism after a bag 
jams the machine. TSA is responsible for EDS and ETD maintenance costs 
after warranties on the machines expire. 14 

From June 2002 through March 2005, Boeing was the prime contractor 
primarily for the installation and maintenance of EDS and ETD machines 
at over 400 U.S. airports. TSA officials stated that the Boeing contract was 
awarded at a time when TSA was a new agency with many demands and 
extremely tight schedules for meeting numerous congressional mandates 
related to passenger and checked baggage screening. The cost 
reimbursement contract 15 entered into with Boeing had been competitively 
bid and contained renewable options through 2007. Boeing subcontracted 
for EDS maintenance through firm-fixed-price contracts 16 with the original 



A TSA official told us that typical EDS warranties are one year and ETD warranties are 
for 2 years. 

15 Cost-reimbursement contracts provide for payment of allowable incurred costs, to the 
extent prescribed in the contract. These contracts establish an estimate of total cost for the 
purpose of obligating funds and establishing a ceiling that the contractor may not exceed 
(except at its own risk) without the approval of the contracting officer. 

16 Firm-fixed-price contracts provide for a price that is not subject to any adjustment on the 
basis of the contractor's cost experience in performing the contract. This contract type 
places upon the contractor maximum risk and full responsibility for all costs and resulting 
profit and loss. It provides maximum incentive for the contractor to control costs and 
perform effectively and imposes a minimum administrative burden upon the contracting 
parties. 
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EDS manufacturers, GE In Vision and L-3, which performed the 
maintenance on their respective EDS. Boeing subcontracted for ETD 
maintenance through a firm-fixed-price contract with Siemens. Consistent 
with language in the fiscal year 2005 House Appropriations Committee 
report and due to TSA's acknowledgment of Boeing's failure to control 
costs, TSA received DHS authorization to negotiate new EDS and ETD 
maintenance contracts in January 2005. 

In March 2005, TSA signed firm-fixed-price contracts for EDS and ETD 
maintenance. TSA awarded a competitively bid contract to Siemens to 
provide maintenance for ETD machines. According to TSA, it negotiated 
sole source contracts with L-3 and GE In Vision for maintaining their 
respective EDS because they are the original equipment manufacturers 
and owners of the intellectual property rights of their respective EDS. In 
September 2005, TSA awarded a competitively bid firm-fixed-price 
contract to Reveal for both the procurement and maintenance of a 
reduced size EDS. 



TSA obligated almost $470 million from fiscal year 2002 through fiscal year 
2005 for EDS and ETD maintenance, according to TSA budget documents. 
In fiscal year 2006, TSA estimates it will spend $199 million and has 
projected it will spend $234 million in fiscal year 2007. 17 According to TSA 
officials, in fiscal year 2004, TSA requested and received approval to 
reprogram about $32 million from another account to EDS/ETD 
maintenance due to higher levels of maintenance costs than expected. 
Similarly, in fiscal year 2005, TSA requested and received approval to 
reprogram $25 million to fund the L-3 contract and to close out the Boeing 
contract. 18 TSA was not able to provide us with data on the maintenance 
cost per machine before fiscal year 2005 because, according to TSA 
officials, TSA's previous contract with Boeing to maintain EDS and ETD 
machines was not structured to capture these data. Table 1 identifies the 



Amounts attributed to maintenance also include utility costs, such as electricity, that 
generally amount to less than 10 percent of the overall amount allocated for maintenance 
each fiscal year, according to TSA officials. Further, TSA officials told us they could 
provide us with amounts obligated for fiscal years 2005 and 2006, but could not provide us 
with the amounts expended for this time period. 

18 As of April 2006, the Boeing contract had yet to be closed out, according to TSA officials. 
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maintenance costs 19 by type of EDS and ETD machine for fiscal years 2005 
and 2006. 


Table 1: Number of EDS and ETD Machines and Annual Per-Machine Maintenance Cost, Fiscal Year 2005 and Fiscal Year 2006 




FY 2005° 




FY 2006 




Type of machine 


Number of 
machines 


Cost per unit 


Number of 
machines 


Cost per unit 


EDS 


GE CTX 2500 


140 


$61,587 


151 


$ 63,590 


GE CTX 5500 


512 


71,549 


547 


73,876 


GE CTX 9000 


172 


93,286 


231 


96,320 


L-3 ex6000 


508 


97,837 


550 


101,000 


Reveal CT-80" 


n/a 


n/a 


16 


n/a 


ETD 


Smiths lonscan 400A 


241 


10,525 


336 


10,974 


Smiths lonscan 400AE 


5 


10,525 


6 


10,974 


Smiths lonscan 400B 


3,038 


8,580 


3,035 


8,946 


Thermo EGIS 3000 


2 


12,899 


2 


13,526 


Thermo EGIS II 


425 


13,134 


545 


13,695 


GE lontrack Itemiser-W 


2,302 


$ 7,727 


2,322 


$ 8,057 



Source: TSA. 

NOTE: Maintenance costs represent the negotiated prices in the maintenance contracts for EDS and 
ETD machines. 

"Fiscal year 2005 per-machine maintenance costs were in effect from March through September 
2005. TSA could not provide per-machine maintenance costs before March 2005. 

"Reveal's EDS machines were installed in fiscal year 2006 and were still under the manufacturer's 
warranty. 

TSA did not provide us with projections of EDS and ETD maintenance 
costs beyond 2007. TSA officials told us that future costs will be 
influenced by the number, type, quantity, and locations of machines 
necessary to support system configurations at airports, such as the extent 
to which EDS are integrated with airport baggage conveyor systems or are 
operated in stand-alone modes. Further, TSA officials told us that future 
EDS and ETD maintenance costs are dependent on decisions related to 
the deployment of new technologies and the refurbishment of existing 
equipment, among other things. The current contracts would have 



19 Represents the negotiated prices for the maintenance of EDS and ETD machines. 
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negotiated maintenance prices per machine through March 2009, if TSA 
decides to exercise option years in the contracts. 

We identified different factors that have played a role in costs to date and 
that will influence future maintenance costs for EDS and ETD machines. 
According to a September 2004 DHS OIG report, TSA did not follow sound 
contracting practices in administering the Boeing contract, which was 
primarily for the installation and maintenance of EDS and ETD machines. 20 
According to DHS OIG officials, TSA's failure to control costs under the 
Boeing contract, including the lack of sound contracting practices, 
contributed to increases in maintenance costs. Among other things, the 
DHS OIG report stated that TSA had paid provisional award fees totaling 
$44 million through December 2003 without any evaluation of Boeing's 
performance. 21 In response to the DHS OIG, TSA agreed to recover any 
excessive award fees paid to Boeing, if TSA determined that such fees 
were not warranted. In commenting on our draft report in July 2006, DHS 
stated that TSA has conducted a contract reconciliation process to ensure 
that no fees would be paid on costs that exceeded the target due to poor 
contractor performance. Further, DHS stated that TSA and Boeing had 
reached an agreement in principle on this matter and that the 
documentation was in the approval process with closure anticipated in 
July 2006. In its report accompanying the DHS Appropriations Bill for 
fiscal year 2007, the House Appropriations Committee stated its need for a 
report from TSA on any actions it has taken to collect excessive award 
fees, how much of the fees have been received to date, and specific plans 
to obligate these collections and cited TSA's plans to use any cost 
recoveries to purchase and install additional EDS. These actions were 
based on the committee's long-standing concerns about the increasing 
costs for EDS and ETD maintenance. 22 In addition to matters related to the 
Boeing contract, TSA officials stated that another factor contributing to 
cost increases were the larger than expected number of machines that 
came out of warranty and their related maintenance costs. According to 
TSA officials, they were not able to determine the cost impact of these 



2U OIG-04-44. 

21 GAO has identified similar instances of agencies' failure to properly use incentives in 
making award fees. See GAO, Defense Acquisitions: DOD Has Paid Billions in Award 
and Incentive Fees Regardless of Acquisition Outcomes, GAO-06-66 (Washington D.C.: 
December 2005). 

22 See H.R. Rep. No. 109-479, at 49-50. 
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additional machines because the Boeing contract was not structured to 
provide maintenance costs for individual machines. 

With regard to future EDS and ETD maintenance costs under firm-fixed- 
price contracts, maintenance costs per machine will increase primarily by 
an annual escalation factor in the contracts that takes into account the 
employment cost index and the consumer price index, 23 if TSA decides to 
exercise contract options. In addition, future maintenance costs may be 
affected by a range of factors, including the number of machines deployed 
and out of warranty, conditions under which machines operate, contractor 
performance requirements, the emergence of new technologies or 
improved equipment, and alternative screening strategies. Lastly, life-cycle 
cost estimates were not developed for the Boeing, Siemens, L-3, and GE 
InVision contracts before the maintenance contracts were executed, and, 
as a result, TSA did not have a sound estimate of maintenance costs for all 
the years the machines are expected to be in operation. In August 2005, 
TSA hired a contractor to define parameters for a life-cycle cost model, 
among other things. This contract states that TSA and the contractor will 
work together to ensure that the full scope of work is planned, 
coordinated, and executed according to approved schedules. In 
commenting on our draft report in July 2006, DHS stated that the TSA 
contractor estimated completing a prototype life-cycle cost model by 
September 2006. Further, DHS stated that TSA's evaluation of the 
prototype would begin immediately upon delivery and that full 
implementation of an EDS life-cycle cost model would be completed 
within 12 months after the prototype had been approved. According to a 
TSA official, the life-cycle cost model would be useful in determining 
machine reliability and maintainability and to inform future contract 
negotiations, such as when to replace a machine versus continuing to 
repair it. 

We identified several actions TSA has taken to control EDS and ETD 
maintenance costs. First, TSA entered into firm-fixed-price contracts 
starting in March 2005 with maintenance contractors, which offer TSA 



For EDS contracts, future labor and material costs could not be determined, so TSA 
negotiated an escalation factor to be used to determine pricing for the contract option 
years. For the ETD contracts, TSA determined after a review of cost data, that it would 
apply a 4 percent escalation factor to prices in the contract option years. The employment 
cost index is a measure of the change in the cost of labor, free from the influence of 
employment shifts among occupations and industries. The consumer price index is a 
measure of the average change in prices over time of goods and services purchased by 
households. 
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certain advantages over cost reimbursement contracts because price 
certainty is guaranteed for up to 5 years if TSA exercises options to 2009. 
Also, TSA included several performance requirements in the Siemens, L-3, 
GE InVision, and Reveal contracts, including the collection of metrics 
related to machine reliability, maintainability, and availability 24 and 
required specific cost data related to maintenance and repair. TSA officials 
told us that these data will assist them in monitoring the contractor 
performance as well as informing future contract negotiations for 
equipment and maintenance. These contracts also stipulate that 
maintenance contractors meet monthly with TSA to review all pertinent 
technical schedules and cost aspects of contracts. TSA also incorporated 
provisions in the L-3 and GE InVision contracts to specify that the agreed 
price for maintaining EDS would be paid only if the contractor performs 
within specified mean downtime (MDT) requirements. 25 Contractors 
submit monthly invoices for 95 percent of the negotiated contract price for 
the month and then submit a MDT report to justify the additional 5 
percent. Consequently, if the contractor fails to fulfill the MDT 
requirements, it is penalized 5 percent of the negotiated monthly 
maintenance price. As of February 2006, neither GE InVision nor L-3 had 
been penalized for missing their MDT requirements. The allowable MDT is 
lowered from 2005 to subsequent renewable years in the contract, as 
shown in table 2. 



Table 2: Mean Downtime Requirement for EDS Contractors, 2005 through 2009 

2005 2006 2007 2008 2009 

24 hours 20 hours 18 hours 14 hours 12 hours 

Source: TSA. 



Includes metrics such as mean time between failures (generally the total time a machine 
is available to perform its required mission divided by the number of failures over a given 
period of time) and operational availability (generally the percentage of time, during 
operational hours, that a machine is available to perform its required mission). Such 
reliability, maintainability, and availability data are standard and appropriate performance 
requirements for maintenance contracts. 

25 As noted in footnote 10, mean downtime is a performance requirement in EDS and ETD 
maintenance contracts. Mean downtime is calculated by the number of hours a machine is 
out of service in a month divided by the number of times that machine is out of service per 
month. For example, if a machine has a total downtime of 50 hours per month and is out of 
service 5 times in that month, the MDT would be equal to 50 divided by 5, which is 10 
hours. 
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With regard to TSA's oversight of EDS and ETD contractor performance, 
TSA's acquisition policies 26 and GAO standards for internal controls 27 call 
for documenting transactions and other significant events, such as 
monitoring contractor activities. The failure of TSA to develop internal 
controls and performance measures has been recognized by other GAO 28 
and DHS OIG reviews. 29 TSA has policies and procedures for monitoring 
its contracts and has included contractor performance requirements in the 
current EDS and ETD maintenance contracts. However, TSA officials 
provided no evidence that they are reviewing maintenance cost data 
provided by the contractor because they are not required to document 
such activities. For example, even though TSA officials told us that they 
are reviewing required contractor data, including actual maintenance costs 
related to labor hours and costs associated with replacing and shipping 
machine parts, they did not have any documentation to support this. TSA 
officials told us that they have begun to capture these data to assist them 
in any future contract negotiations. 

Further, TSA officials provided no evidence that performance data for 
corrective and preventative maintenance required under contracts are 
being reviewed. TSA officials told us that they perform such reviews, but 
do not document their activities since there are no TSA policies or 
procedures requiring them to do so. Therefore, TSA could not provide 
assurance that contractors are complying with contract performance 
requirements. For example, although TSA documents monthly meetings 
with contractors to discuss performance data, TSA officials did not 
provide evidence that they independently determine the reliability and 
validity of data required by the contracts, such as mean time between 
failures and mean time to repair, which are important to making informed 
decisions about future purchases of EDS and ETD equipment and their 
associated maintenance costs. Further, TSA officials provided no evidence 



TSA uses the Federal Aviation Administration (FAA) Acquisition Management System 
which, as adopted by TSA, requires contractors to act on contractual quality assurance 
commitments and ensure that government quality and reliability needs are met (FAA 
Acquisition Management Policy 3.10.4.2). 

27 GAO/AIMD-00-21.3.1. 

28 GAO has identified contract surveillance issues in other agencies, such as the 
Department of Defense. See GAO, Contract Management: Opportunities to Improve 
Surveillance on Department of Defense Service Contracts, GAO-05-274 (Washington, D.C.: 
March 2005). 

29 OIG-04-44. 
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that they ensure that contractors are performing scheduled preventative 
maintenance. TSA officials told us that they review the contractor- 
submitted data to determine whether contractors are fulfilling their 
contractual obligations, but do not document their activities because there 
are no TSA policies or procedures to require such documentation. 

Additionally, for EDS contracts with possible financial penalties, TSA 
officials told us that they review contractor-submitted mean downtime 
data on a monthly basis to determine the reliability and validity of the data 
and to determine whether contractors are meeting contract provisions or 
should be penalized. However, TSA officials do not document these 
activities because there are no TSA policies or procedures requiring them 
to do so. As a result, without adequate documentation, there is no 
assurance as to whether or not contractors are meeting contract 
provisions or that TSA has ensured that it is making appropriate payments 
for services provided. 



The cost of maintaining checked baggage-screening equipment has 
increased as more EDS and ETD machines have been deployed and 
warranties expire. TSA's move in March 2005 to firm-fixed-price 
maintenance contracts for EDS and ETD maintenance was advantageous 
to the government in that it helps control present and future maintenance 
costs. Firm-fixed-price contracts also help ensure price certainty and 
therefore are more predictable. However, unresolved issues remain with 
the past contractor, specifically fees awarded to former contractor Boeing 
that may have been excessive due to a lack of timely evaluation of the 
contractor's performance. The House Appropriations Committee has 
expressed concern about these unresolved issues; specifically, what 
actions TSA has taken to recover these excessive fees, and the extent to 
which any collections might impact future TSA obligations. Closing out the 
Boeing contract is essential to resolving these issues. In responding to our 
draft report, DHS stated that the completion of an EDS life-cycle cost is 
over a year away. Absent such a life-cycle cost model, TSA may not be 
identifying cost efficiencies and making informed procurement decisions 
regarding the future purchase of EDS and ETD machines and maintenance 
contracts. Further, TSA must provide evidence of its reviews and analyses 
of contractor-submitted data and perform analyses of contractor data to 
determine the reliability and validity of the data and to provide assurance 
of compliance with contract performance requirements and internal 
control standards. Without stronger oversight, TSA will not have 
reasonable assurance that contractors are performing as required and that 
full payment is justified based on meeting mean downtime requirements. 



Conclusions 
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R6COmm6ndcltiori.S ^° i m P rove TSA's management of EDS and ETD maintenance costs 

" and strengthen oversight of contract performance, we recommend that the 
Secretary of Homeland Security instruct the Assistant Secretary, 
Transportation Security Administration, to take the following three 
actions: 



• establish a timeline to complete its evaluation and close out the Boeing 
contract and report to congressional appropriations committees on its 
actions, including any necessary analysis, to address the Department of 
Homeland Security Office of Inspector General's recommendation to 
recover any excessive fees awarded to Boeing Service Company; 

• establish a timeline for completing life-cycle cost models for EDS, 
which TSA recently began; and 

• revise policies and procedures to require documentation of the 
monitoring of EDS and ETD maintenance contracts to provide 
reasonable assurance that 

• contractor maintenance cost data and performance data are 
recorded and reported in accordance with TSA contractual 
requirements and 

• self-reported contractor mean downtime data are valid, reliable, and 
justify the full payment of the contract amount. 



Agency Comments 
and Our Evaluation 



We provided a draft of this report to DHS for its review and comment. On 
July 24, 2006, we received written comments on the draft report. DHS, in 
its written comments, concurred with our findings and recommendations, 
and agreed that efforts to implement these recommendations are essential 
to a successful explosive detection systems program. DHS stated that it 
has initiated efforts to improve TSA's management of EDS and ETD 
maintenance costs and strengthen oversight of contract performance. 
Regarding our recommendation that TSA establish a timeline to close out 
the Boeing contact and report to congressional committees on its actions 
to recover any excessive fees, DHS stated that TSA has conducted a 
contract reconciliation process to ensure that no fees would be paid on 
costs that exceeded the target due to poor contractor performance and 
that Boeing and TSA have reached an agreement in principle on this 
matter and the documentation is in the approval process with closure 
anticipated in July 2006. Regarding our recommendation to establish a 
timeline for completing the EDS life-cycle cost model, DHS stated that 
TSA expects to complete its prototype evaluation in September 2006 and 
that the EDS life-cycle cost model will be completed 12 months after the 
prototype has been approved. Regarding our recommendation to revise 
TSA policies and procedures to require documentation of its monitoring of 
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EDS and ETD maintenance contracts, DHS stated that a TSA contractor is 
developing automated tools to perform multiple analyses of contractor- 
submitted data that DHS said would allow TSA to accurately and 
efficiently certify the contractors' performance against their contractual 
requirements and would allow TSA to independently validate and verify 
maintenance and cost data. The department's comments are reprinted in 
appendix II. 



We will send copies of this report to the Secretary of Homeland Security 
and the Assistant Secretary, Transportation Security Administration, and 
interested congressional committees. We will also make copies available 
to others upon request. In addition, the report will be available at no 
charge on GAO's Web site at http://www.gao.gov. If you or your staffs have 
any questions or need additional information, please contact me at (202) 
512-8777 or berrickc@gao.gov. Contact points for our Offices of 
Congressional Relations and Public Affairs may be found on the last page 
of this report. Key contributors to this report are acknowledged in 
appendix III. 




Cathleen A. Berrick 

Director, Homeland Security and Justice Issues 
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Explosive Detection Systems 
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Information for Congressional Committees 

April 19, 2006 
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Introduction 



House Conference Report 109-241 — which accompanied the 
Department of Homeland Security Appropriations Act, 2006 
(Public Law 109-90) 1 — directed GAO to report on the reasons 
for cost increases in maintaining TSA's explosive detection 
systems, 2 including TSA's related contracting practices. In 
February 2006, a House Appropriations staffer told us that this 
information would assist in the committee's budget 
deliberations. 



'H.R. Conf. Rep. No. 109-241, at 52 (2005). 

Z TSA interprets the term explosive detection system to include both explosive detection systems (EDS) and explosive trace detection (ETD) machines. 
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Objectives 



• What are the historical, current, and projected costs for the 
maintenance of explosive detection systems (EDS) and 
explosive trace detection (ETD) machines? 

• What factors played a role in EDS and ETD maintenance 
costs and what factors could affect future costs? 

• What has TSA done to control EDS and ETD maintenance 
costs? 

• To what extent does TSA oversee the performance of EDS 
and ETD maintenance contractors? 
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Scope and Methodology 



To determine TSA costs to maintain EDS and ETD machines we 

• reviewed TSA contract files and budget documents for fiscal years 2003 through 2007, and 

• interviewed TSA headquarters officials, Department of Homeland Security Office of the Inspector General 
(DHS OIG) officials, and EDS and ETD contractor representatives. For purposes of our review, we 
focused on the amounts obligated under contracts to maintain the machines. We did not review TSA's 
negotiations for maintenance services or the process for awarding contracts, nor did we assess other 
direct or indirect costs related to TSA or DHS employees engaged in contract administration or other 
related items. 

To determine what factors played a role in maintenance costs and what TSA has done to control costs, we: 

• reviewed TSA contract files, acquisition and strategic plans, budget documents, TSA processes for 
reviewing contract cost and performance data, and a DHS OIG report; 1 and 

• interviewed TSA headquarters officials, DHS OIG officials, and EDS and ETD contractor representatives. 

To determine the extent of TSA contract oversight, we: 

• reviewed TSA contract files and processes for reviewing contract performance data, 

• interviewed TSA headquarters officials and EDS and ETD contractor representatives, and 

• reviewed GAO standards for internal controls 2 and our previous work on TSA's acquisition function. 3 

We performed our work from January 2006 through June 2006 in accordance with generally accepted 
government auditing standards. 



1 U.S. Department of Homeland Security, Office of Inspector General, Evaluation of TSA's Contract for the Installation and Maintenance of Explosive 
Detection Equipment at United States Airports, OIG-04-44 (Washington, D.C.: September 2004). 

2 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21 .3.1 (Washington, D.C.: November 1 999). 

Transportation Security Administration: High-Level Attention Needed to Strengthen Acquisition Function, GAO-04-544 (Washington, D.C.: May 2004). 
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• According to TSA budget documents, TSA has obligated almost $470 
million from fiscal year 2002 through fiscal year 2005 for EDS and ETD 
maintenance. In fiscal year 2006, TSA estimates it will spend $199 million 
and has projected it will spend $234 million in fiscal year 2007. 1 

• TSA was unable to provide us data on maintenance cost per machine 
prior to fiscal year 2005 because, according to TSA officials, its 
previous contract with Boeing Service Company (Boeing) to maintain 
EDS and ETD machines was not structured to capture these data. 

• TSA did not provide us with projections of EDS and ETD maintenance 
costs beyond fiscal year 2007, although TSA has negotiated 
maintenance prices per machine through fiscal year 2009. 



1 Amounts attributed to maintenance also include utility costs, such as electricity, that generally amount to less than 1 percent of the overall amount 
allocated for maintenance each fiscal year, according to TSA officials. Further, TSA officials could provide us with amounts obligated for fiscal years 2005 
and 2006, but could not provide us with the amounts expended for this time period. 
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Results in Brief 



• Different factors have played a role in costs to date and will influence future maintenance costs: 

• According to the DHS OIG, TSA had awarded excessive fees under the Boeing contract, 
which ran from June 2002 to March 2005. TSA agreed to recover any excessive fees paid 
to Boeing, if TSA determines that such fees are not warranted. However, as of April 2006, 
TSA had not yet completed its evaluation of Boeing's performance and thus had not yet 
determined if there are any fees to recover. TSA officials stated that they did not know 
when TSA would make a final determination on this matter. 

• TSA officials stated that cost increases were due, in part, to the larger than expected 
number of machines that came out of warranty. 

• Lifecycle cost estimates were not developed for Boeing, or for Siemens Maintenance 
Services (Siemens), L-3 Communications Security and Detection Systems (L-3), and 
General Electric InVision, Inc. (GE InVision), which also have maintenance contracts with 
TSA. However, in August 2005, TSA contracted for the development of a lifecycle cost 
model, including cost estimates, which TSA officials say will assist them in managing future 
maintenance costs. In February 2006, the contractor had begun work on the model, 
however, TSA did not know when the model would be completed. 
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Results in Brief 



• TSA's move to firm fixed price contracting in March 2005 provided 
advantages over cost-reimbursement type contracts, such as price 
certainty. Additionally, TSA included performance requirements in 
contracts, including financial penalties in some contracts. 

• Although TSA has included contractor performance requirements in the 
EDS and ETD maintenance contracts, some oversight issues remain. For 
example, although TSA officials told us that they review the contractor- 
submitted data to determine whether contractors are fulfilling their 
contractual obligations, such as mean downtime data, they do not 
document their activities because there are no TSA policies or procedures 
to require such documentation. As a result, there is no assurance as to 
whether or not contractors are meeting contract provisions or should be 
penalized. 
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Results in Brief 



We are making three recommendations to the Secretary of Homeland Security to help improve 
TSA's management of EDS and ETD maintenance costs and oversight of contractor 
performance. We are recommending that: 

• TSA report to the congressional appropriations committees on its actions, including any 
necessary analysis, to address the DHS OIG recommendation to recover any excessive 
fees awarded to Boeing; 

• TSA establish a timeline to complete lifecycle cost models for EDS and ETD machines, 
which TSA recently began; and 

• TSA revise its polices and procedures to document its monitoring of EDS and ETD 
maintenance contracts to provide reasonable assurance that contractor performance data 
are recorded and reported in accordance with TSA contractual requirements and self- 
reported contractor mean downtime is valid, reliable and justify the full payment of the 
contractor amount. 

TSA reviewed these slides in their entirety and provided several technical comments, which we 
incorporated as appropriate. 
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Background 



TSA was mandated to screen all checked baggage using explosive 
detection systems at airports by December 31 , 2003. 1 

Explosive Detection Systems (EDS) use computer-aided 
tomography X-rays to recognize the characteristics of explosives. 
In general, EDS are used for checked baggage screening. 

Explosive Trace Detection (ETD) machines use chemical analysis 
to detect traces of explosive material vapors or residues. ETD 
machines are used for both passenger carry-on baggage and 
checked baggage screening. 

According to TSA budget documents, TSA will have deployed over 
1,400 EDS and 6,600 ETD machines at baggage screening 
locations in over 400 airports nationwide by the end of fiscal year 
2006. 



1 The Aviation and Transportation Security Act, Pub. L. No. 1 07-71 § 1 1 0(b), 1 1 5 Stat. 597, 61 5 (2001 ) mandated, among other things, that all checked baggage at 
U.S. airports be screened using explosive detection systems by December 31 , 2002. Section 425 of the subsequently enacted Homeland Security Act of 2002, 
7-296, 1 1 6 Stat. 21 35, 21 85-86, in effect, extended this mandate to December 31 , 2003. See 49 U.S.C. § 44901 (d). 
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Background 



• TSA is responsible for the EDS and ETD maintenance costs 
after their warranties expire. 1 

• EDS and ETD maintenance includes preventative 
maintenance — scheduled activities to increase machine 
reliability that are performed monthly, quarterly, and yearly 
based on the contractors' maintenance schedules and 
corrective maintenance — actions performed to restore 
machines to operating condition after failure. 



1 A TSA official told us that typical EDS warranties are for one year and that ETD warranties are for 2 years. 



10 
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Background — TSA Cost Reimbursement 
Contract with Boeing 

• From June 2002 through March 2005, Boeing was the prime contractor for the installation and 
maintenance of EDS and ETD machines at over 400 U.S. airports. TSA officials stated that the 
Boeing contract was awarded at a time when TSA was a new agency with many demands and 
extremely tight schedules for meeting numerous congressional mandates related to passenger 
and checked baggage screening. Boeing had a cost reimbursement contract 1 with TSA, which 
was competitively bid and contained renewable options to 2007. 

• Boeing subcontracted EDS maintenance through firm fixed price contracts 2 with the original 
EDS manufacturers, GE InVision and L-3, which performed maintenance on their 
respective EDSs. 

• Boeing subcontracted ETD maintenance through a firm fixed price contract with Siemens. 

• Consistent with language in the fiscal year 2005 DHS House Appropriations Committee 
report and due to TSA's acknowledgment of Boeing's failure to control costs, TSA received 
DHS authorization to negotiate new EDS and ETD maintenance contracts in January 2005. 



1 Cost reimbursement contracts provide for payment of allowable incurred costs, to the extent prescribed in the contract. These contracts establish an estimate of total cost for the purpose of 
obligating funds and establishing a ceiling that the contractor may not exceed (except at its own risk) without approval of the contracting officer. -t 
2 Firm fixed price contracts provide for a price that is not subject to any adjustment on the basis of the contractor's cost experience in performing the contract. This contract type places upon 
the contractor maximum risk and full responsibility for all costs and resulting profit and loss. It provides maximum incentive for the contractor to control costs and perform effectively and 
imposes a minimum administrative burden upon the contracting parties. 
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Background — TSA Firm Fixed Price Contracts 
with Siemens, L-3, and GE InVision 

• In March 2005, TSA signed firm fixed price contracts for EDS and ETD 
maintenance. 

• TSA awarded a competitively bid contract to Siemens to provide maintenance 
for ETD machines. 

• TSA negotiated sole source contracts with L-3 and GE InVision because they 
are the original equipment manufacturers and owners of the intellectual 
property of their respective EDS. 

• TSA can exercise 4 1-year options on all three contracts through March 2009. 

• In September 2005, TSA awarded a competitively bid firm fixed price contract to 
Reveal Imaging Technologies, Inc., (Reveal) for both the procurement and 
maintenance of a reduced-size EDS. 
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Historical, Current, and Projected Costs for 
EDS and ETD Maintenance 



13 
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EDS and ETD Maintenance Costs 



According to TSA budget documents, TSA 
has obligated almost $470 million for EDS 
and ETD maintenance from fiscal years 
2002 through 2005. 
Overall, costs for EDS and ETD 
maintenance grew from $14 million in 
fiscal year 2002 to an estimated $199 
million in fiscal year 2006. In fiscal year 
2007, TSA projects it will spend $234 
million. 

TSA was not able to provide us data on 
the maintenance cost per machine prior to 
fiscal year 2005 because, according to 
TSA officials, its previous contract with 
Boeing to maintain EDS and ETD 
machines was not structured to capture 
these data. 

According to TSA officials, in fiscal year 
2004, TSA requested and received 
approval to reprogram about $32 million 
due to higher levels of maintenance costs 
than expected. 

In fiscal year 2005, TSA requested and 
received approval to reprogram $25 million 
to fund the L-3 contract ($16.6 million) and 
to closeout the Boeing contract ($8.4 
million), which has yet to be closed. 



EDS and ETD Machine Maintenance Budget Amounts, Fiscal Years 2002 through 2007 





Fiscal 


Amount 


Appropriated 


Prior year 


Re- 


Total 


Amount 


year 


Requested 


(as revised) 


carry over 


programmed 


available 


Obligated 


2002" 












14 


2003 


75 


75 


25 




100 


93 


2004 


100 


75 


61 


32 


168 


168 


2005 


205 


175 


5 


25 


205 


195 


2006 b 


200 


199 






199 


199 


2007 


234 













Source: TSA. 

Note: TSA's budgeted amounts for EDS and ETD maintenance include utility costs for the 
operation of the machines. Such costs comprise less than 10 percent of the total amounts 
budgeted for maintenance each fiscal year. According to TSA, in fiscal year 2002 and fiscal 
year 2003, the amount for utilities was negligible. GAO did not independently verify budget 
amounts. 



According to TSA officials, in fiscal year 2002, TSA was budgeted approximately $527 million 
for EDS equipment installation and maintenance, of which $14 million was expended for 
maintenance. TSA carried forward fiscal year 2002 funding for EDS and ETD maintenance to 
fiscal year 2003 and fiscal year 2004. 

'Fiscal year 2006 obligations are estimated. 
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EDS and ETD Maintenance Costs 



• TSA officials did not provide us with projections of costs 
beyond 2007. However, current contracts have negotiated 
maintenance prices per machine through March 2009, if TSA 
decides to exercise option years in the contracts. 

• Future EDS and ETD maintenance costs depend on 
decisions made as outlined in a February 2006 TSA strategic 
planning framework for screening checked baggage using 
EDS and ETD. Among other things, the plan discusses 
options for the deployment of new technologies and 
refurbishment of existing equipment. 
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Factors That Played a Role in EDS and ETD 

Maintenance Costs 
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Factors That Played a Role in Maintenance 
Costs and Could Impact Future Costs 



• Different factors have played a role in costs to date and will influence future maintenance costs for EDS 
and ETD machines: 



• According to a September 2004 DHS OIG report, TSA did not follow sound contracting practices in 
administering the Boeing contract, which was primarily for the installation and maintenance of EDS and 
ETD machines. 1 

• Among other things, the DHS OIG found that TSA had paid provisional award fees totaling $44 
million through December 2003 without any evaluation of Boeing's performance. 2 

• In response to the DHS OIG, TSA agreed to recover any excessive award fees paid to Boeing if 
TSA determines that such fees are not warranted. However as of April 2006, TSA had not yet 
completed its evaluation of Boeing's performance and thus had not yet determined if there were 
any fees to recover. 

• TSA officials told us that they did not know when TSA would make a final determination on this 
matter. 

• TSA officials stated that cost increases were also due to the larger than expected number of machines 
that came out of warranty. 



'OIG-04-44. 

2 GAO has identified similar instances of agencies' failure to properly use incentives in making award fees. See GAO, Defense Acquisitions: DOD Has Paid Billions in 
Award and incentive Fees Regardless of Acquisition Outcomes, GAO-06-66 (Washington, D.C.: December 2005). 
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Factors That Played a Role in Maintenance 
Costs and Could Impact Future Costs 



• Under the current firm fixed price contracts, if TSA exercises 
4 1-year options through March 2009, maintenance costs per 
machine would increase primarily by an annual escalation 
factor in the contracts that takes into account the employment 
cost index and the consumer price index. 1 



1 For EDS contracts, future labor and material costs could not be determined, so TSA negotiated an escalation factor to be used to determine pricing for the contract option years. For the ETD 
contracts, TSA determined after a review of cost data, that it would apply a 4 percent escalation factor to prices in the contract option years. The employment cost index is a measure of the "j 8 
change in the cost of labor, free from the influence of employment shifts among occupations and industries. The consumer price index is a measure of the average change in prices over time 
s purchased by households. 
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Factors That Played a Role in Maintenance 
Costs and Could Impact Future Costs 

• Future maintenance costs will be impacted by a range of factors, including the number of 
machines deployed and out of warranty, conditions under which machines operate, mean 
downtime requirements, the emergence of new technologies or improved equipment, and 
alternative screening strategies. 

• TSA's February 2006 strategic plan framework for screening checked baggage over 
the next 20 years discusses factors that may impact future maintenance costs. For 
example, the framework discusses the refurbishment of existing machines and the 
deployment of new technologies, but does not outline the number of machines or 
specific time frames for implementation. Additionally, the impact of these strategies 
on future maintenance costs is unknown. 1 

• If no new equipment or maintenance providers emerge, TSA may pay a premium in 
future sole source contracts where intellectual property rights are involved. For 
example, because L-3 and GE InVision had intellectual property rights on their 
machines, their maintenance contracts were not bid competitively and therefore 
prices were not subject to the benefits of market forces. 



'TSA issued its strategic plan framework for screening checked baggage using EDS and ETD machines in response to various 
congressional mandates, congressional committee directives, and GAO recommendations. 
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Factors That Played a Role in Maintenance 
Costs and Could Impact Future Costs 

• Lifecycle cost estimates were not developed for the Boeing, 
Siemens, L-3, and GE contracts before the maintenance 
contracts were executed and, as a result, TSA did not have a 
complete picture of all maintenance costs. In August 2005, 
TSA hired a contractor to define parameters for a lifecycle 
cost model. A TSA official told us that the contractor began 
work on a lifecycle cost model for EDS in February 2006 and 
did not know when the model would be completed. 
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TSA Actions to Control 
EDS and ETD Maintenance Cost Increases 
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TSA Actions to Control EDS and ETD 
Maintenance Cost Increases 



Firm fixed price contracts starting in 
March 2005 offer TSA certain 
advantages over cost 
reimbursement type contracts: 

• Price certainty is guaranteed 
for up to five years if TSA 
exercises options to 2009. 

• Siemens and Reveal 
contracts were competitively 
bid, although the GE InVision 
and L-3 contracts were sole 
source contracts. 

TSA did not provide per-unit 
maintenance costs prior to March 
2005 because the Boeing contract 
was not structured to capture these 
data. 



Number of EDS and ETD Machines and Annual Per Machine Maintenance Cost, 
Fiscal Year 2005 and Fiscal Year 2006 





FY 2005 a 




FY 2006 






Number of 


Cost 


Number of 


Cost 


Type of machine 


machines 


per unit 


machines 


per-unit 


EDS 


GECTX2500 


140 


$61 ,587 


151 


$ 63,590 


GECTX5500 


512 


71 ,549 


547 


73,876 


GE CTX 9000 


172 


93,286 


231 


96,320 


L-3 ex6000 


508 


97,837 


550 


101,000 


Reveal CT-80 b 


n/a 


n/a 


16 


n/a 




ETD 


Smiths lonscan 400A 


241 


10,525 


336 


10,974 


Smiths lonscan 400AE 


5 


10,525 


6 


10,974 


Smiths lonscan 400B 


3,038 


8,580 


3,035 


8,946 


Thermo EGIS 3000 


2 


12,899 


2 


13,526 


Thermo EGIS II 


425 


13,134 


545 


13,695 


GE lontrack Itemiser-W 


2,302 


$ 7,727 


2,322 


$ 8,057 



Source: TSA. 

NOTE: Maintenance costs represent the negotiated prices in the maintenance contracts for EDS and 
ETD machines. 

a Fiscal year 2005 per-machine maintenance costs were in effect from March through September 

2005. TSA could not provide per machine maintenance costs prior to March 2005. 

b Reveal EDS were installed in fiscal year 2006 and were still under the manufacturer's warranty. 
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TSA Actions to Control EDS and ETD 
Maintenance Cost Increases 

• TSA included several contractor performance requirements in the 
Siemens, L-3, GE InVision, and Reveal contracts. 

• Metrics related to Reliability, Maintainability, and Availability 
(RMA) of the machines must be reported to TSA. 1 

• Specific cost data related to maintenance and repair must be 
reported to TSA. 

• Contractors are required to meet monthly with TSA to review all 
pertinent technical, schedule, and cost aspects of the contract, 
including an estimate of the work to be accomplished in the 
next month; performance measurement information; and any 
current and anticipated problems. 



1 1ncludes metrics such as m ean time between failures (generally the total time a machine is available to perform its required mission divided by the num ber of 23 

failures over a given period of time) and operational availability (generally the percentage of time, during operational hours, that a machine is available to perform 

its required mission). Such reliability, maintainability, and availability data are standard and appropriate performance requirements for maintenance contracts. 
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TSA Actions to Control EDS and ETD 
Maintenance Cost Increases 

• Provisions in the L-3 and GE InVision contracts specify that the agreed price for maintaining EDS will 
be paid only if the contractor performs within specified mean downtime (MDT) requirements. 

• MDT is calculated by the number of hours a machine is out of service in a month divided by 
the number of times that machine is out of service per month. 1 

• Contractors submit monthly invoices for 95 percent of the negotiated contract price for the 
month and then submit an MDT report to justify the additional 5 percent. Consequently, if the 
contractor fails to fulfill the MDT requirements, it is penalized 5 percent of the negotiated 
monthly maintenance price. 

• As of February 2006, neither GE InVision nor L-3 have been penalized for missing their MDT. 

• The allowable MDT is lowered from 2005 to subsequent renewable years in the contract, as 
shown in the table below. 



Mean Downtime Requirement for EDS Contractors, 2005 through 2009 



2005 


2006 


2007 


2008 


2009 


24 hours 


20 hours 


18 hours 


14 hours 


12 hours 



1 For example, if a machine has a total downtime of 50 hours per month and is out of service 5 times in that month, the MDT would be equal to 50 divided by 5, 
which is 1 hours. 24 
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TSA Oversight of 
EDS and ETD Contractor Performance 
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TSA Oversight of EDS and ETD 
Contractor Performance 



• TSA's acquisition policies 1 and GAO's standards for internal controls 2 call for documenting transactions and 
other significant events, such as monitoring contractor activities. Although TSA has policies and procedures 
for monitoring their contracts and TSA has included contractor performance requirements in the current EDS 
and ETD maintenance contracts, some oversight issues remain. The failure of TSA to develop internal 
controls and performance measures has been recognized by other GAO and DHS OIG reviews. 3 



• TSA officials provided no evidence that they are reviewing maintenance cost data provided by the 
contractor because they are not required to document such activities. For example, even though TSA 
officials told us they are reviewing required contractor data, including actual maintenance costs related 
to labor hours, costs associated with replacement parts, and the costs of shipping machine parts, they 
did not have any documentation to support this. TSA officials told us that they have begun to capture 
these data to assist them in any future contract negotiations. 



• TSA officials provided no evidence that performance data for corrective and preventative maintenance 
required under the contract is being reviewed. TSA officials told us that they perform such reviews, but 
do not document their activities since there are no TSA policies or procedures requiring them to do so. 
Therefore, TSA could not provide assurance that contractors are complying with contract performance 
requirements. For example, although TSA documents monthly meetings with contractors to discuss 
performance data, TSA did not provide evidence that it independently determines the reliability and 
validity of data required by the contracts, such as mean time between failures and mean time to repair, 
which are important to making informed decisions about future purchases of EDS and ETD equipment 
and their associated maintenance costs. 



'TSA uses the Federal Aviation Administration (FAA) Acquisition Management System which, as adopted by TSA, requires contractors to act on contractual quality assurance commitments and ensure that government 
quality and reliability needs are met (FAA Acquisition Management Policy 3.10.4.2). r 
2 GAO/AIMD-00-21.3.1. £ 
3 GAO has identified contract surveillance issues in other agencies, such as the Department ol Defense. See GAO, Contract Management: Opportunities to Improve Surveillance on Depanment ot Defense Service 
Contracts, GAO-05-274 (Washington, D.C.: March 2005). See also OIG-04-44. 
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TSA Oversight of EDS and ETD 
Contractor Performance 

• For EDS contracts with possible financial penalties, TSA officials 
told us that they review contractor-submitted mean downtime data 
on a monthly basis to determine the reliability and validity of the 
data and to determine whether contractors are meeting contract 
provisions or should be penalized. However, TSA officials said they 
do not document these activities because there are no TSA policies 
or procedures to do so. 

• As a result, without adequate documentation, there is no assurance 
on whether contractors are meeting contract provisions that TSA 
has ensured that it is making appropriate payments for services 
provided. 
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Conclusions 



• TSA's move to firm fixed price maintenance contracts was advantageous to the 
government in that it helps control present and future maintenance costs. Firm fixed price 
contracts also help ensure price certainty and therefore are more predictable. 

• Unresolved issues remain with the past contractor, specifically fees awarded to former 
contractor Boeing that may have been excessive due to a lack of timely evaluation of the 
contractor's performance. 

• Although TSA has begun to develop a lifecycle cost model in order to control costs and 
negotiate future contracts, TSA has not set a timeframe to complete this model. Without 
such a time frame, TSA may not be identifying cost efficiencies and making informed 
procurement decisions. 

• Further, TSA must provide evidence of its reviews and analyses of contractor-submitted 
data and perform analyses of contractor data to determine the reliability and validity of the 
data and to provide assurance of contractor compliance with contract performance 
requirements and internal control standards. Without stronger oversight, TSA will not have 
reasonable assurance that contractors are performing as required and that full payment is 
justified based on meeting mean downtime requirements. 
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Recommendations 



• To help improve TSA's management of EDS and ETD maintenance costs and 
strengthen oversight of contract performance, we recommend that the Secretary of 
Homeland Security instruct the Assistant Secretary, Transportation Security 
Administration to take the following three actions 

• report to the congressional appropriations committees on its actions, including any 
necessary analysis, to address the DHS OIG recommendation to recover any 
excessive fees awarded to Boeing; 

• establish a time line for completing a lifecycle cost model for EDS, which TSA 
recently began; and 

• revise its policies and procedures to require documentation of its monitoring of EDS 
and ETD maintenance contracts to provide reasonable assurance that 

• contractor maintenance cost data and performance data are recorded and 
reported in accordance with TSA contractual requirements and 

• self-reported contractor mean downtime data are valid, reliable, and justify the 
full payment of the contract amount. 
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Agency Comments 



TSA reviewed these slides in their entirety and provided several 
technical comments, which we incorporated as appropriate. TSA 
officials told us that they are not making formal comments on our 
recommendations. 
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U.S. Department of Homeland Security 

Washington, DC 20528 



Homeland 
Security 



July 24, 2006 



Ms. Cathleen A. Berrick 

Director, Homeland Security and Justice Issues 

U.S. Government Accountability Office 

441 G Street, NW 

Washington, DC 20548 

Dear Ms. Berrick: 

Thank you for the opportunity to comment on draft report GAO-06-795, "Oversight of Explosive 
Detection Systems Maintenance Contracts Can Be Strengthened." The Department of Homeland 
Security (DHS) concurs with the recommendations, and appreciates the time and resources that 
GAO has devoted to this important review. The findings of this report will help improve TSA's 
management of Explosives Detection Systems (EDS) and Explosives Trace Detection (ETD) 
maintenance costs and strengthen oversight of contract performance. TSA believes that these 
recommendations are essential to a successful EDS maintenance program and has already 
initiated efforts in response to the recommendations outlined in the audit. In addition, we have 
comments in a few areas. 

In March 2005, TSA entered into firm fixed price contracts for maintenance of EDS and ETD 
machines. This method of contracting is highly advantageous to the government because price 
certainty is guaranteed. Firm fixed price contracts help control present and future maintenance 
costs and ensure that costs are more predictable. This method of contracting will assist TSA in 
controlling costs. An additional method for controlling cost is the life cycle cost model. TSA 
has begun to develop a life cycle cost model in order to control costs and negotiate future 
contracts. The life cycle cost model will be used to determine machine reliability and 
maintainability and influence future contract negotiations, such as when to replace a machine 
versus continuing to repair it. The prototype Life Cycle Cost Model is estimated to be completed 
by September 2006. 

The report states that TSA was unable to provide GAO with data on the per machine 
maintenance costs prior to fiscal year 2005. The Boeing contract was not structured to capture 
this data. While the Boeing contract did not require submission of data deliverables that 
identified the costs per machine, TSA did receive cost data during the course of the contract to 
support the negotiations and develop a follow-on strategy. 



www.dhs.gov 



Page 47 



GAO-06-795 TSA EDS Maintenance Costs 



Appendix II: Agency Comments 



2 



Regarding the Boeing contract, TSA reaffirms that the total potential fee was awarded to Boeing 
under a competitive environment and determined it to be fair and reasonable for the work 
performed. The requirement to deploy electronic screening equipment to 440+ airports in an 
abbreviated timeframe set forth an extremely challenging environment for the contractor. 
Cost, schedule, and performance parameters were hurried and lacked significant definition. 
TSA does not intend to seek reimbursement of fees associated with the total fee percentage. 
TSA conducted a contract reconciliation process to ensure that no fees would be paid on costs 
that exceeded the target due to poor contractor performance (i.e. cost overrun). Boeing and TSA 
have reached an agreement in principle on this matter and the documentation is in the approval 
process with closure anticipated in July 2006. 

Finally, TSA established contract performance requirements in March 2005 to evaluate and 
monitor contractor performance regarding EDS and ETD maintenance. These requirements are 
included in the current maintenance contracts with Siemens, L-3, GE InVision, and Reveal: 
1) metrics related to Reliability, Maintainability, and Availability (RMA) of the machines must 
be reported to TSA; 2) specific cost data related to maintenance and repair must be reported to 
TSA; 3) contractors are required to meet monthly with TSA to review all pertinent technical, 
schedule, and cost aspects of the contract, including an estimate of the work to be accomplished 
in the next month, performance measurement information, and any current and anticipated 
problems. 

This information is stored in the RMA database which tracks RMA and data requirements for all 
maintenance contracts. TSA is developing automated tools to perform multiple analyses of 
maintenance data which will allow TSA to spot and track cost and performance trends as they 
occur. TSA's independent validation and verification of maintenance and cost data will allow it 
to accurately and efficiently assess the maintenance contractors' performance against contract 
requirements. 

TSA appreciates your review of the EDS maintenance program and thanks GAO for the 
thorough analysis and discussion that comprises this report. We will continue to re-evaluate our 
processes in line with best business practices and use the findings and recommendations of this 
report as a reminder of the task before us. 

The following represents our responses to the recommendations. 

Recommendation 1: Establish a timeline to close out the Boeing contract and report to 
congressional committees on its actions, including any necessary analysis, to address the 
DHS OIG recommendation to recover any excessive fees awarded to Boeing. 

Concur. TSA awarded the total potential fee to Boeing under a competitive environment 
determined to be fair and reasonable for the work performed. The requirement to deploy 
electronic screening equipment to 440+ airports in an abbreviated timeframe set forth an 
extremely challenging environment for the contractor. Cost, schedule, and performance 
parameters were hurried and lacked significant definition. While the Boeing contract did not 
require submission of data deliverables that identified the costs per machine, TSA did receive 
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cost data to support the negotiations and development of a follow-on strategy. TSA does not 
intend to seek reimbursement of fees associated with the total fee percentage. TSA conducted a 
contract reconciliation process to ensure that no fees would be paid on costs that exceeded the 
target due to poor contractor performance. Boeing and TSA have reached an agreement in 
principle on this matter and the documentation is in the approval process with closure anticipated 
in July 2006. 

Recommendation 2: Establish a timeline for completing the lifecycle cost model for 
Explosive Detection Systems. 

Concur. TSA has begun to develop a life cycle cost model in order to control costs and 
negotiate future contracts. The life cycle cost model will be used to determine machine 
reliability and maintainability and inform future contract negotiations, such as when to replace a 
machine versus continuing to repair it. The prototype Life Cycle Cost Model is in development, 
with an estimated completion date of September 2006. Prototype evaluation will begin 
immediately upon delivery. Full implementation of an EDS Life Cycle Cost Model will be 
completed within 12 months after the prototype has been approved. 

Recommendation 3: Revise the policies and procedures to require documentation of its 
monitoring of EDS and ETD maintenance contracts to provide reasonable assurance that 
contractor maintenance cost data and performance data are recorded and reported in 
accordance with TSA contractual requirements and self reported contractor mean 
downtime data are valid, reliable, and justify the full payment of the contract amount. 

Concur. In March 2005, TSA established specific procedures, which are included in all 
maintenance contracts, to evaluate and monitor contractor performance with regard to EDS and 
ETD maintenance. There are three specific contractor performance requirements that are 
included in the Siemens, L-3, GE In Vision, and Reveal contracts: 1) metrics related to 
Reliability, Maintainability, and Availability (RMA) of the machines must be reported to TSA; 
2) specific cost data related to maintenance and repair must be reported to TSA; 3) contractors 
are required to meet monthly with TSA to review all pertinent technical, schedule, and cost 
aspects of the contract, including an estimate of the work to be accomplished in the next month, 
performance measurement information, and any current and anticipated problems. 

TSA established the RMA database, simply called RMA. This system tracks RMA and data 
requirements for all maintenance contracts. In addition, TSA has tasked its Integrated Logistics 
Support (ILS) contractor to track and review all reporting and data deliveries from the 
maintenance contractors. The ILS contractor has developed software tools for tracking and 
managing all deliverable products, to include submission dates and completeness of submission. 
The ILS contractor is developing automated tools to perform multiple analyses of the digital data 
delivered by the maintenance contractors, to include a visual dashboard that will allow TSA to 
spot and track cost and performance trends as they occur. This independent validation and 
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verification of maintenance and cost data will allow TSA to accurately and efficiently certify the 
maintenance contractors' performance against their contract requirements. 

Thank you for the opportunity to review and provide comments to the draft report. 



Sincerely, 



Steven J. Pecinovsky 

Director Departmental GAO/OIG Liaison Office 
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